Endogenous Macrodynamics in Algorithmic Recourse

IEEE Conference on Secure and Trustworthy Machine Learning

Patrick Altmeyer

Delft University of Technology

Giovan Angela
Aleksander Buszydlik
Karol Dobiczek
Arie van Deursen
Cynthia C. S. Liem

February 7, 2023

Background

  • Counterfactual Explanation (CE) explain how inputs into a model need to change for it to produce different outputs.
  • Counterfactual Explanations that involve realistic and actionable changes can be used for the purpose of Algorithmic Recourse (AR) to help individuals who face adverse outcomes.

Example: Consumer Credit

In Figure 1, arrows indicate changes from factuals (credit denied) to counterfactuals (credit supplied).

Figure 1: Counterfactuals for Give Me Some Credit dataset (Kaggle 2011).

Our work in a nutshell …

[…] we run experiments that simulate the application of recourse in practice using various state-of-the-art counterfactual generators and find that all of them induce substantial domain and model shifts.

Figure 2 illustrates how the application of recourse can induce shifts.

Figure 2: Dynamics in Algorithmic Recourse.

Proof-of-Concept

Figure 3: A bank has trained a model to evaluate credit applicants. Credit risk is highest in bottom-right corner.

Figure 4: The bank has provided recourse to unsuccessful applicants: endogenous domain shift.

Figure 5: The bank has retrained the classifier: endogenous model shift.

Figure 6: The outcome after the process has been repeated a few times. Average default risk has increased.

Questions …

  • Who should bear the risk?
  • Are the counterfactuals genuinely valid in practice?
  • What about fairness and privacy concerns?

Experiments

Empirical Setup

Evaluation: we propose metrics to measure domain shifts and model shifts.

Models: we use linear classifiers, deep neural networks and deep ensembles.

Data:

Synthetic
  • Overlapping, Linearly Separable, Circles, Moons.

Real-World

Generators: Wachter (Wachter, Mittelstadt, and Russell 2017), REVISE (Joshi et al. 2019), DiCE (Mothilal, Sharma, and Tan 2020), Greedy (Schut et al. 2021)

Principal Findings — Synthetic

Domain shifts for overlapping synthetic data using deep ensemble.

Performance deterioration for overlapping synthetic data using deep ensemble.

Principal Findings — Real-World

Model shifts for Credit Default data using deep ensemble.

Performance deterioration for Credit Default data using deep ensemble.

Gradient-Based Recourse Revisited

From Individual Recourse …

Many existing approaches to CE and AR work with the following baseline:

\[ \begin{aligned} \color{lightgrey} \mathbf{s}^\prime &\color{lightgrey}= \arg \min_{\mathbf{s}^\prime \in \mathcal{S}} \{ {\text{yloss}(M(f(\mathbf{s}^\prime)),y^*)} \\&\color{black} + \lambda {\text{cost}(f(\mathbf{s}^\prime)) } \color{lightgrey} \} \end{aligned} \tag{1}\]

Typically concern has centred around minimizing costs to a single individual.

… towards collective recourse

We propose to extend Equation 1 as follows:

\[ \begin{aligned} \color{lightgrey}\mathbf{s}^\prime &\color{lightgrey}= \arg \min_{\mathbf{s}^\prime \in \mathcal{S}} \{ {\text{yloss}(M(f(\mathbf{s}^\prime)),y^*)} \\ &\color{lightgrey}+ \lambda_1 {\text{cost}(f(\mathbf{s}^\prime))} + \color{black}\lambda_2 {\text{extcost}(f(\mathbf{s}^\prime))} \color{lightgrey}\} \end{aligned} \tag{2}\]

  • The newly introduced term \(\text{extcost}(f(\mathbf{s}^\prime))\) is meant to explicitly capture external costs generated by changes to \(\mathbf{s}^\prime\).

Mitigation Strategies

  1. More Conservative Decision Thresholds
  2. Classifier Preserving ROAR (ClaPROAR)1
  3. Gravitational Counterfactual Explanations

Figure 7: Mitigation strategies.

Secondary Findings

Domain shifts for overlapping synthetic data using deep ensemble.

Performance deterioration for overlapping synthetic data using deep ensemble.

Discussion

Key Takeaways 🔑

  • State-of-the-art approaches to AR induce substantial domain and model shifts.
  • External costs of Individual Recourse should be shared across stakeholders.
  • Straightforward way to achieve this is to explicitly penalize external costs in the counterfactual search objective function (Equation 2).

Future Research Ideas

Private vs. External Costs:

  • How can we make informed choices about the tradeoff between private and external costs?
  • Pareto-optimal collective recourse?

Causal Modelling:

  • How do things play out for recent approaches to AR that incorporate causal knowledge such as Karimi, Schölkopf, and Valera (2021)?

Counterfactual Explanations and Probabilistic Machine Learning

Methodologies and open-source tools that help researchers and practitioners assess the trustworthiness of predictive models.

Towards Trustworthy AI in Julia
  1. CounterfactualExplanations.jl (JuliaCon 2022)
  2. ConformalPrediction.jl (JuliaCon 2023 — I hope!)
  3. LaplaceRedudx.jl (JuliaCon 2022)
  4. AlgorithmicRecourseDynamics.jl

… contributions welcome! 😊

📚 More Reading

Image Sources

  • Copyright for stock images belongs to TU Delft.
  • All other images, graphics or animations were created by us.

References

IEEE Conference on Secure and Trustworthy Machine Learning ’23 — Patrick Altmeyer — CC BY-NC

Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.
Kaggle. 2011. “Give Me Some Credit, Improve on the State of the Art in Credit Scoring by Predicting the Probability That Somebody Will Experience Financial Distress in the Next Two Years.” Kaggle. https://www.kaggle.com/c/GiveMeSomeCredit.
Karimi, Amir-Hossein, Bernhard Schölkopf, and Isabel Valera. 2021. “Algorithmic Recourse: From Counterfactual Explanations to Interventions.” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 353–62.
Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17.
Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.
Upadhyay, Sohini, Shalmali Joshi, and Himabindu Lakkaraju. 2021. “Towards Robust and Reliable Algorithmic Recourse.” https://arxiv.org/abs/2102.13620.
Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841.
Yeh, I-Cheng, and Che-hui Lien. 2009. “The Comparisons of Data Mining Techniques for the Predictive Accuracy of Probability of Default of Credit Card Clients.” Expert Systems with Applications 36 (2): 2473–80.